{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 1. Photoshop-like Functions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#Table of Contents\n", "* [1. Photoshop-like Functions](#1.-Photoshop-like-Functions)\n", "\t* [1.1 Green screen Effects - Chroma Key Compositing](#1.1-Green-screen-Effects---Chroma-Key-Compositing)\n", "\t* [1.2 Obamicon](#1.2-Obamicon)\n", "\t* [1.3 Edge Detection](#1.3-Edge-Detection)\n", "\t* [1.4 Blurring](#1.4-Blurring)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1.1 Green screen Effects - Chroma Key Compositing" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "https://en.wikipedia.org/wiki/Chroma_key\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Displaying an Image\n", "\n", "This example introduces 6 new things related to images:\n", "\n", "* Processing Javascript Directives: /* @pjs preload=...; */\n", "* PImage img;\n", "* img = loadImage(FILENAME);\n", "* img.resize(WIDTH, HEIGHT);\n", "* img.width, img.height\n", "* image(img, X, Y);\n", "\n", "First, we download an image to use:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloaded 'greenscreen1.jpg'.\n" ] } ], "source": [ "%download http://blog.unem.de/wp-content/uploads/2012/05/11-Greenscreen-Clarissa-Knorr.jpg -f greenscreen1.jpg" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we display it:" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_19\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_19\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_19\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_19\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #19:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #19 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "/* @pjs preload=\"greenscreen1.jpg\"; */\n", "\n", "PImage img;\n", "\n", "void setup() {\n", " img = loadImage(\"greenscreen1.jpg\");\n", " // println(\"\" + img.width + \" x \" + img.height)\n", " // 1920 x 1080\n", " // Resize to a more reasonable size:\n", " img.resize(int(img.width/10.0), int(img.height/10.0));\n", " // Make sketch size match:\n", " size(img.width, img.height);\n", "}\n", "\n", "void draw() {\n", " image(img, 0, 0);\n", " noLoop();\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Accessing each Pixel in a PImage\n", "\n", "We can get each pixel/color in an image by using these:\n", "\n", "* img.loadPixels();\n", "* color c = img.pixels[x + y * img.width];\n", "* red(c), green(c), blue(c)" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_20\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_20\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_20\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_20\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #20:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #20 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "/* @pjs preload=\"greenscreen1.jpg\"; */\n", "\n", "PImage img;\n", "\n", "void setup() {\n", " img = loadImage(\"greenscreen1.jpg\");\n", " // println(\"\" + img.width + \" x \" + img.height)\n", " // 1920 x 1080\n", " // Resize to a more reasonable size:\n", " img.resize(int(img.width/10.0), int(img.height/10.0));\n", " // Make sketch size match:\n", " size(img.width, img.height);\n", " // Add this to get pixel information into array:\n", " img.loadPixels();\n", "}\n", "\n", "void draw() {\n", " image(img, 0, 0);\n", " noLoop();\n", "}\n", "\n", "void mousePressed() {\n", " // Make sure the index into pixels is an int:\n", " color pixel = img.pixels[int(mouseX + mouseY * img.width)];\n", " // See what color it is:\n", " println(\"Color: \" + red(pixel) + \", \" + green(pixel) + \", \" + blue(pixel));\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Composite Images\n", "\n", "In addition, we can also change pixels in an image by:\n", "\n", "* img.pixel[x + y * width] = new_color;\n", "* img.updatePixels();" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloaded 'background1.jpg'.\n" ] } ], "source": [ "%download http://images.tenplay.com.au/~/media/News/National%20News/National%20News%20do%20not%20delete/News_National_showheader_480x210.jpg -f background1.jpg" ] }, { "cell_type": "code", "execution_count": 37, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_21\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_21\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_21\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_21\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #21:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #21 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "/* @pjs preload=\"greenscreen1.jpg,background1.jpg\"; */\n", "\n", "PImage img;\n", "PImage background;\n", "PImage composite;\n", "\n", "void setup() {\n", " img = loadImage(\"greenscreen1.jpg\");\n", " img.resize(int(img.width/10.0), int(img.height/10.0));\n", " size(img.width, img.height);\n", " // get background:\n", " background = loadImage(\"background1.jpg\");\n", " background.resize(img.width, img.height);\n", " // Now we image processing:\n", " composite = new PImage(img.width, img.height);\n", " background.loadPixels();\n", " img.loadPixels();\n", " composite.loadPixels();\n", " for (int i = 0; i < composite.width; i++) {\n", " for (int j = 0; j < composite.height; j++) {\n", " // if pixel is green, get it from background\n", " // else get it from foreground\n", " if (i > composite.width/2) {\n", " composite.pixels[int(i + j * composite.width)] = \n", " img.pixels[int(i + j * img.width)];\n", " } else {\n", " composite.pixels[int(i + j * composite.width)] = \n", " background.pixels[int(i + j * img.width)];\n", " }\n", " }\n", " }\n", " composite.updatePixels();\n", "}\n", "\n", "void mousePressed() {\n", " color pixel = composite.pixels[int(mouseX + mouseY * img.width)];\n", " println(\"Color: \" + \n", " red(pixel) + \", \" + \n", " green(pixel) + \", \" + \n", " blue(pixel));\n", "}\n", "\n", "void draw() {\n", " image(composite, 0, 0);\n", " noLoop();\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Composite each pixel based on color" ] }, { "cell_type": "code", "execution_count": 38, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_22\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_22\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_22\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_22\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #22:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #22 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "/* @pjs preload=\"greenscreen1.jpg,background1.jpg\"; */\n", "\n", "PImage img;\n", "PImage background;\n", "PImage composite;\n", "\n", "void setup() {\n", " img = loadImage(\"greenscreen1.jpg\");\n", " img.resize(int(img.width/10.0), int(img.height/10.0));\n", " size(img.width, img.height);\n", " // get background:\n", " background = loadImage(\"background1.jpg\");\n", " background.resize(img.width, img.height);\n", " // Now we image processing:\n", " composite = new PImage(img.width, img.height);\n", " background.loadPixels();\n", " img.loadPixels();\n", " composite.loadPixels();\n", " for (int i = 0; i < composite.width; i++) {\n", " for (int j = 0; j < composite.height; j++) {\n", " color pixel = img.pixels[int(i + j * img.width)];\n", " float r = red(pixel);\n", " float g = green(pixel);\n", " float b = blue(pixel);\n", " // if pixel is green, get it from background\n", " if (r < 55 && \n", " g < 170 && g > 145 && \n", " b > 50 && b < 80) {\n", " composite.pixels[int(i + j * composite.width)] = background.pixels[int(i + j * img.width)];\n", " } else {\n", " // else get it from foreground\n", " composite.pixels[int(i + j * composite.width)] = pixel;\n", " }\n", " }\n", " }\n", " composite.updatePixels();\n", "}\n", "\n", "void mousePressed() {\n", " color pixel = img.pixels[int(mouseX + mouseY * img.width)];\n", " println(\"Color: \" + red(pixel) + \", \" + green(pixel) + \", \" + blue(pixel));\n", "}\n", "\n", "void draw() {\n", " image(composite, 0, 0);\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Problem 1**: fix the green screen effect above to make the background a nice, uninterrupted image without any green showing through.\n", "\n", "**Problem 2**: Make your own \"green screen\" effect using your own pictures. Use a colored blanket, for example, as your background. Do something other than a news cast simulation. For example, put yourself in an unexpected place." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1.2 Obamicon" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloaded 'obama.jpg'.\n" ] } ], "source": [ "%download http://i29.photobucket.com/albums/c290/trebor007/image_3655004.jpg -f obama.jpg" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Problem 3**: Take the regular picture of Obama and see if you can produce one programmatically that looks like the artistic version.\n", "\n", "**Problem 4**: Take a pictures of yourself, and make it look like an Obamicon-ified version." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1.3 Edge Detection" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "https://processing.org/examples/edgedetection.html\n", "\n", "There are a series of image processing techniques that can be accomplished by comparing a pixel to its surrounding 8 pixels.\n", "\n", "A sample kernel for edge detection (high-pass filter):\n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", " \n", " \n", " \n", "\n", "
-1-1-1
-1+9-1
-1-1-1
\n", "\n", "```java\n", "sum = pixel1 * k[-1][-1] + \n", " pixel2 * k[0][-1] +\n", " pixel3 * k[1][-1] +\n", " pixel4 * k[-1][0] +\n", " pixel5 * k[0][0] +\n", " pixel6 * k[1][0] +\n", " pixel7 * k[-1][1] +\n", " pixel8 * k[0][1] +\n", " pixel9 * k[1][1];\n", "// set color at pixel5 to be sum\n", "```" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloaded 'moon.jpg'.\n" ] } ], "source": [ "%download https://processing.org/examples/moon.jpg" ] }, { "cell_type": "code", "execution_count": 39, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_23\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_23\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_23\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_23\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #23:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #23 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "// The next line is needed if running in JavaScript Mode with Processing.js\n", "/* @pjs preload=\"moon.jpg\"; */\n", "\n", "float[][] kernel = {{ -1, -1, -1}, \n", " { -1, 9, -1}, \n", " { -1, -1, -1}};\n", " \n", "PImage img;\n", "\n", "void setup() { \n", " size(640, 360);\n", " img = loadImage(\"moon.jpg\"); // Load the original image\n", "}\n", "\n", "void draw() {\n", " image(img, 0, 0); // Displays the image from point (0,0) \n", " img.loadPixels();\n", " // Create an opaque image of the same size as the original\n", " PImage edgeImg = createImage(img.width, img.height, RGB);\n", " // Loop through every pixel in the image.\n", " for (int y = 1; y < img.height-1; y++) { // Skip top and bottom edges\n", " for (int x = 1; x < img.width-1; x++) { // Skip left and right edges\n", " float sum = 0; // Kernel sum for this pixel\n", " for (int ky = -1; ky <= 1; ky++) {\n", " for (int kx = -1; kx <= 1; kx++) {\n", " // Calculate the adjacent pixel for this kernel point\n", " int pos = (y + ky)*img.width + (x + kx);\n", " // Image is grayscale, red/green/blue are identical\n", " float val = red(img.pixels[pos]);\n", " // Multiply adjacent pixels based on the kernel values\n", " sum += kernel[ky+1][kx+1] * val;\n", " }\n", " }\n", " // For this pixel in the new image, set the gray value\n", " // based on the sum from the kernel\n", " edgeImg.pixels[int(y * img.width + x)] = color(sum, sum, sum);\n", " }\n", " }\n", " // State that there are changes to edgeImg.pixels[]\n", " edgeImg.updatePixels();\n", " image(edgeImg, width/2, 0); // Draw the new image\n", " noLoop();\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Problem 5**: Try edge detection on one of your own images, but with a purpose. What is the use of edge detection?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 1.4 Blurring" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "https://processing.org/examples/blur.html\n", "\n", "A sample kernel for blurring (low-pass filter):\n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", " \n", " \n", " \n", "\n", "
1/91/91/9
1/91/91/9
1/91/91/9
" ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_24\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_24\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_24\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_24\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #24:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #24 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "// The next line is needed if running in JavaScript Mode with Processing.js\n", "/* @pjs preload=\"moon.jpg\"; */ \n", "\n", "float v = 1.0 / 9.0;\n", "float[][] kernel = {{ v, v, v }, \n", " { v, v, v }, \n", " { v, v, v }};\n", " \n", "PImage img;\n", "\n", "void setup() {\n", " size(640, 360);\n", " img = loadImage(\"moon.jpg\"); // Load the original image\n", "} \n", "\n", "void draw() {\n", " image(img, 0, 0); // Displays the image from point (0,0) \n", " img.loadPixels();\n", "\n", " // Create an opaque image of the same size as the original\n", " PImage edgeImg = createImage(img.width, img.height, RGB);\n", "\n", " // Loop through every pixel in the image\n", " for (int y = 1; y < img.height-1; y++) { // Skip top and bottom edges\n", " for (int x = 1; x < img.width-1; x++) { // Skip left and right edges\n", " float sum = 0; // Kernel sum for this pixel\n", " for (int ky = -1; ky <= 1; ky++) {\n", " for (int kx = -1; kx <= 1; kx++) {\n", " // Calculate the adjacent pixel for this kernel point\n", " int pos = (y + ky)*img.width + (x + kx);\n", " // Image is grayscale, red/green/blue are identical\n", " float val = red(img.pixels[pos]);\n", " // Multiply adjacent pixels based on the kernel values\n", " sum += kernel[ky+1][kx+1] * val;\n", " }\n", " }\n", " // For this pixel in the new image, set the gray value\n", " // based on the sum from the kernel\n", " edgeImg.pixels[int(y * img.width + x)] = color(sum);\n", " }\n", " }\n", " // State that there are changes to edgeImg.pixels[]\n", " edgeImg.updatePixels();\n", "\n", " image(edgeImg, width/2, 0); // Draw the new image\n", " noLoop();\n", "}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Problem 6**: Perform blurring on one of your own images. Why might this be useful?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Bonus**: combine two of the photoshop-like effects in a single image." ] }, { "cell_type": "code", "execution_count": 49, "metadata": { "collapsed": false }, "outputs": [ { "data": { "application/javascript": [ "\n", " var component = document.getElementById(\"sketch_32\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"state_32\");\n", " if (component != undefined)\n", " component.remove();\n", " component = document.getElementById(\"controls_div_32\");\n", " if (component != undefined)\n", " component.remove();\n", " require([window.location.protocol + \"//calysto.github.io/javascripts/processing/processing.js\"], function() {\n", " // FIXME: Stop all previously running versions (?)\n", " var processingInstance = Processing.getInstanceById(\"canvas_32\");\n", " if (processingInstance != undefined && processingInstance.isRunning())\n", " processingInstance.noLoop();\n", " });\n", " " ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "
\n", " Sketch #32:
\n", "
\n", "
\n", "
\n", " \n", " \n", " \n", " \n", "
\n", "Sketch #32 state: Loading...
\n", "\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "/* @pjs preload=\"pinkscreen.png\"; */ \n", "\n", "PImage img;\n", "PImage composite;\n", "PImage moon;\n", "\n", "void setup() {\n", " img = loadImage(\"pinkscreen.png\");\n", " img.resize(img.width/2, img.height/2);\n", " size(img.width, img.height);\n", " composite = new PImage(img.width, img.height)\n", "}\n", "\n", "void draw() {\n", " image(img, 0, 0);\n", "}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Calysto Processing", "language": "java", "name": "calysto_processing" }, "language_info": { "codemirror_mode": { "name": "text/x-java", "version": 2 }, "file_extension": ".java", "mimetype": "text/x-java", "name": "java" } }, "nbformat": 4, "nbformat_minor": 0 }